The prediction error of autoregressive small sample models

A fundamental problem in order selection is that one single realization of a stochastic process is used twice, for the estimation of parameters for different model orders and for the selection of the best model order. Parameters are estimated by the minimization of the residual variance; higher model orders with more estimated parameters will always give a smaller residual variance. The purpose of order selection is to find the model order that gives the best fit to other re-alizations of the same stochastic process. This fit is expressed by the squared prediction error and it will increase if too many parameters are used. The weak parameter criterion (WPC) is an estimate for the squared prediction error, with as special feature that it is computed from the same observations that are used for the estimation of the parameters .